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BACKGROUND OF THE INVENTION 
1. The Field of the Invention 

0001 The present invention relates to the field of network communications. More 
specifically, the present invention relates to the real-time communication of multimedia 
information over heterogeneous networks. 



2. The Related Art 

O 0002 Multimedia information includes information that is efficiently interpretable by 

ffi one or more of the five human senses, but mostly by the human senses of sight and 

YJ: hearing. For example, video information is interpreted by the senses of sight and hearing. 

'% Audio information is interpreted by the sense of hearing. Some user interfaces such as 

j^n Braille displays present information for interpretation by the sense of touch. However, 

y, with the advancement of appropriate user interfaces, multimedia information may also 

Q include taste and smell information as well. 

0003 Often, multimedia information is time-sensitive and should be rendered at the 

^ same speed that the information is sampled within some minimal jitter tolerances. This 

m 

§3 ^ type of multimedia presentation will often be referred herein as "real-time". For efficient 

^ H ^ J 00 

& o 2 g 2 5j real-time delivery of multimedia data, the networks between the multimedia source and the 



W £ -l H U 



S <j S 2 o & multimedia sink should be abstracted so that the networks as a whole function as a constant 



^ S < I S h delay network. 



< 4 



0004 Figure 1 illustrates a conventional network 100 for delivering multimedia data 



^ in real-time. A multimedia source 101 transmits multimedia packets 104 over a constant 

delay network 103 to a multimedia sink 102. Of course, there is always some variance in 
the delay, however small, introduced by any network. However, a "constant delay" 
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network is a network in which the delay variance introduced by the network is below the 
minimal jitter tolerances required to support real-time presentation of the multimedia data. 

0005 As illustrated in Figure 1, the constant delay network 103 may include a 
number of different network types that follow different standards. Nevertheless, in order 
to support real-time multimedia data delivery, each component network must support the 
abstraction of the networks as a whole into the constant delay network. There are 
generally two classifications of networks that support this kind of abstraction into an 
amalgamated constant delay network, 

0006 One such classification is constant delay networks. Thus, for example, the 
constant delay network 103 of Figure 1 includes a component constant delay network 105. 
Constant delay networks receive multimedia data from a point on the network at a certain 
receive rate, and deliver the multimedia data to another point on the network at the same 
rate so that there is a relatively constant delay. A second such classification is variable 
delay networks that have a common notion of time. 

0007 It is possible even in such variable delay networks to emulate a constant delay 
network if the network supports a common notion of time across the network. For 
example, the IEEE 1394 serial bus is not inherently a constant delay network. However, 
the IEEE 1394 serial bus does support a common notion of time. The IEC 61883-x 
standard uses the common notion of time present in the IEEE 1394 serial bus to have the 
IEEE 1394 serial bus emulate a constant delay network to within the jitter tolerances 
required under the MPEG-2 standard. 

0008 Referring to Figure 1, the constant delay network 103 also may include a 
variable delay network 106. If a series of one or more contiguous variable delay networks 
is interposed between the multimedia source 101 and the multimedia sink 104, the series of 
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variable delay networks may emulate a constant delay network if the string of variable 
delay networks shares a common notion or reckoning of time. 

0009 Thus, conventional methods allow for the real-time delivery of multimedia data 
over a network that includes both constant delay networks and variable delay networks that 
have a common notion of time. However, there is a third classification of networks that is 
not compatible with being a component network in an amalgamated constant delay 
network under conventional standards. That third classification is a variable delay network 
that does not have a common notion of time. 

0010 Under conventional technology, real-time communication of multimedia data is 
not possible if one or more of the networks that must be traversed are variable delay 
networks that do not have a common notion of time. Accordingly, there exists no 
conventional infrastructure for seamlessly communicating multimedia over heterogeneous 
networks in real-time since those heterogeneous networks may include one or more 
variable delay networks that contain no common time reference across the network. 
Establishing such an infrastructure would allow for better access to multimedia content 
regardless of the heterogenic nature of the networks that intervene between the multimedia 
source and the multimedia sink. 
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SUMMARY OF THE INVENTION 

0011 The present invention extends to the real-time communication of multimedia 
data over heterogeneous networks that may include constant delay networks, variable delay 
networks that have a common reckoning of time, and variable delay networks that do not 
have a common reckoning of time. Currently, real-time communication of multimedia 
data may be accomplished point-to-point over specific networks. However, there is no 
conventional way of communicating such data in real-time over heterogeneous networks 
that include variable delay networks that have no common reckoning of time. 

0012 If there are any variable delay networks in the heterogeneous networks in which 
there is no common reckoning of time, a common reckoning of time is established in each 
of those networks. Then, the variable delay networks are emulated as one or more constant 
delay networks using the specific common time reckoning present in each variable delay 
network. Thus, the principles of the present invention allow for the real-time 
communication of multimedia information over heterogeneous networks. By so doing, a 
user may more flexibly access multimedia data in real-time regardless of the location of the 
individual. 

0013 In one aspect of the present invention, a constant delay network is emulated 
over a variable delay network despite the fact that the transmitter includes a transmitter 
application that interfaces with a link layer device driver over a variable delay interface. 
The transmitter application provides multimedia packets for transmission over the variable 
delay network. In addition, the transmitter application provides the multimedia packets 
with time stamps that represent the time according to the reckoning of the transmitter 
application. When the transmitter link layer device receives the multimedia packet, it 
calculates a new time stamp that is in accordance with a network time base, the new time 
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stamp being based on the time stamp provided by the transmitter application. The receiver 
link layer controller then uses the new time stamp to evaluate the rendering time of the 
corresponding information in the packet. 

0014 Another aspect of the invention permits for communication over a variable 
delay network that does not inherently have a time base. Instead, the transmitter 
application periodically transmits a current time to various receiver devices on the network 
in order to synchronize the devices on the network. Then, the transmitter includes a time 

13 

stamp that follows the synchronized time in order to permit the information in the 

m 

y multimedia packets to be presented at the appropriate time. 

IS 0015 Additional features and advantages of the invention will be set forth in the 

B f* description, which follows, and in part will be obvious from the description, or may be 

^ learned by the practice of the invention. The features and advantages of the invention may 

1 1 be realized and obtained by means of the instruments and combinations particularly 

\2 pointed out in the appended claims. These and other features of the present invention will 

become more fully apparent from the following description and appended claims, or may 

^ be learned by the practice of the invention as set forth hereinafter. 

k4 
w 
ua 
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BRIEF DESCRIPTION OF THE DRAWINGS 

0016 In order that the manner in which the above-recited and other advantages and 
features of the invention are obtained, a more particular description of the invention briefly 
described above will be rendered by reference to specific embodiments thereof, which are 
illustrated, in the appended drawings. Understanding that these drawings depict only 
typical embodiments of the invention and are not therefore to be considered to be limiting 
of its scope, the invention will be described and explained with additional specificity and 
detail through the use of the accompanying drawings in which: 

0017 Figure 1 schematically illustrates a constant delay network that supports real- 
time multimedia communication in accordance with the prior art; 

0018 Figure 2 illustrates an exemplary system that provides a suitable operating 
environment for the present invention; 

0019 Figure 3 schematically illustrates multimedia communication over a 
l*T heterogeneous network that may include constant delay networks, variable delay networks 

with a common network time base, and variable delay networks without a common 
h network time base; 

^ _ 0020 Figure 4 illustrates a flowchart of a method of performing constant delay 

p3 I $ g § 3j communication of multimedia packets over a heterogeneous network; 

g i& % 8 £ 0021 Figure 5 illustrates the heterogeneous network of Figure 3 in which the variable 



^S<§«^ delay networks that did not have a common network time base are provided with a 

o 



common network time base; 
^ 0022 Figure 6 illustrates the heterogeneous network of Figure 5 in which the variable 

delay networks emulate constant delay networks using the common network time base; 
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0023 Figure 7 illustrates a variable delay network in which there is an undedicated 
variable delay interface between the transmitter application and the transmitter link layer 
controller as when the variable delay interface is a PCI interface; 

0024 Figure 8 illustrates a method for emulating a constant delay network in a 
variable delay network even if there is a variable delay interface between the transmitter 
application and the transmitter link layer controller; 

0025 Figure 9 illustrates a variable delay network in which a common time base is 
established; and 

0026 Figure 10 illustrates a method of emulating a constant delay network using a 
variable delay network that is not conventionally equipped with a common network time 
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DETAILED DESCRIPTION OF THE INVENTION 

0027 The present invention extends to the real-time communication of multimedia 
data over heterogeneous networks that may include constant delay networks, variable delay 
networks that have a common reckoning of time, and variable delay networks that do not 
have a common reckoning of time. If there are any variable delay networks in which there 
is no common reckoning of time, a common reckoning of time is established in each of 
those networks. Then, the common reckonings of times may be used to emulate the 
variable delay networks as one or more constant delay networks. 

0028 The embodiments of the present invention may comprise a special purpose or 
!;« general-purpose processing device or computer including various computer hardware 
j* components, as discussed in greater detail below. The embodiments may further comprise 
□ multiple computers linked in a networked environment. Set top boxes that enhance the 
U capabilities of conventional televisions represent an example of a special purpose 

; .\. i 

Q computer. 

0029 Embodiments within the scope of the present invention also include computer- 
readable media for carrying or having computer-executable instructions or data structures 

i 

g „ stored thereon. Such computer-readable media can be any available media that can be 

§q i 3 8 * 5 accessed by a general purpose or special purpose computer. By way of example, and not 
O ° «> < P >; 

g 1 1 jjj | g limitation, such computer-readable media can comprise physical storage media such as 

« 1 5 1 g £ RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or 

^ other magnetic storage devices, or any other medium that can be used to carry or store 

^ desired program code means in the form of computer-executable instructions or data 
structures and that can be accessed by a general purpose or special purpose computer. 
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0030 When information is transferred or provided over a network or another 
communications connection (either hardwired, wireless, or a combination of hardwired or 
wireless) to a computer, the computer properly views the connection as a computer- 
readable medium. Thus, such a connection is also properly termed a computer-readable 
medium. Combinations of the above should also be included within the scope of 
computer-readable media. Computer-executable instructions comprise, for example, 
instructions and data which cause a general purpose computer, special purpose computer, 
or special purpose processing device to perform a certain function or group of functions. 

0031 The invention will be described in the general context of computer-executable 
instructions, such as program modules, being executed by set-top boxes or other 
computers. Generally, program modules include routines, programs, objects, components, 
data structures, etc. that perform particular tasks or implement particular abstract data 
types. The sequence of instructions implemented in a particular data structure or program 
module represents examples of corresponding acts for implementing the functions or steps 
described herein. 

0032 Figure 2 and the corresponding discussion are intended to provide a general 
description of a suitable environment in which the invention may be implemented. In the 
discussion, reference is made to a home entertainment system that may be used for 
displaying and/or recording programming. For purposes of this description and in the 
claims, a "home entertainment system" may be a display unit, such as a television screen, 
coupled to a processing device for performing the data processing acts and steps disclosed 
herein, or may include any number of interconnected consumer electronic devices, one of 
which having a processing device for performing the data processing disclosed herein. 
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0033 Examples of such consumer electronic devices include a video cassette recorder 
("VCR"), a video game system, a stereo system, a television or monitor with data 
processing capabilities, a cable television box, a digital satellite system receiver ("DSS"), a 
digital video broadcasting system ("DVB"), a digital versatile disc system ("DVD"), a 
compact disk read-only memory system ("CD-ROM"), a set-top box that serves as an 
Internet terminal, and any other device capable of processing data as described herein. 
Furthermore, the term "home entertainment system" is to be understood as a term that 
broadly describes a television-viewing or music listening environment, whether it is 
located in a viewer's home, at a place of business, in the public, or at any other location. 
Also for purposes of this description and in the claims, the term "programming" includes 
both the viewable and non-viewable portions of moving image data and/or its associated 
sound data. 

0034 In one embodiment, the present invention is implemented in a system that uses a 
conventional television screen or other display unit to display information and includes a 
WebTV® set-top box or a similar Internet terminal that has been adapted to perform the 
operations that include composing, sending and receiving email, browsing the World Wide 
Web ("Web"), accessing other segments of the Internet, and otherwise displaying 
information. An Internet terminal may use standard telephone lines, Integrated Services 
Digital Network (ISDN) lines, cable lines associated with cable television service, or the 
like to connect to the Internet or other wide area networks. 

0035 Figure 2 illustrates a home entertainment system 210 that includes a 
management system 212, a display device 214 and an audio system 216. Management 
system 212 may be a set-top box or Internet terminal that has been adapted to perform the 
operations disclosed herein. Management system 212 may be integrally positioned with or 
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separate from display device 214, which may be a high definition television display, a 
standard television display, a flat panel display, a projection device, an interface involving 
direct neural stimulation, a computer monitor, or any other device capable of displaying 
viewable video image data. Audio system 216 may be a speaker, a stereo system, or any 
device capable of emitting sound data, and similarly may be integrally positioned with or 
separate from display device 214. 

0036 Management system 212 includes a signal input 218, which receives 
programming from a signal source 220. The programming is transmitted from signal 
source 220 to signal input 218 via a programming input line 222, which can be a cable or 
go optic connection, a terrestrial antenna system, a satellite system, or any device or system 

£ capable of transmitting programming to home management system 212. 

Q 0037 The signal source 220 may be either a single channel signal source or a multiple 

f * channel signal source. A single channel signal source provides programming from a 

j *f recorded medium, such as a videocassette, compact disc, etc. Examples of a single channel 

signal source include a VCR, a DVD, and the like, Alternatively, a multiple channel signal 
>t source includes any system or device that is capable of sending a signal that may be 

si 

^ „ received by a satellite receiver, a cable or optic connection, a terrestrial antenna, or the 

£j 1 5 8 ^e. Examples of a multiple channel signal source include DSS/DVB, a cable box, locally 

£j i [3 2 o E broadcast programming (i.e. programming broadcast using UHF or VHF), and the like. 



0038 While Figure 2 illustrates home entertainment system 210 as having a single 
programming input line 222 and a single signal source 220, there may also be a plurality of 



^ programming input lines that transmit programming from a plurality of signal sources. In 

such embodiments, the home entertainment system may receive the programming from 
one signal source or from a plurality of signal sources at a time. 
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0039 Management system 212 also includes a user input interface 224, which 
receives input from an input device 226, such as a remote control, external special purpose 
or general-purpose processing device or computer, keyboard, microphone, mouse, or any 
other device capable of generating electronic instructions for management system 212. 
Input device 226 is communicatively coupled to management system 212 over an input 
link 228 so as to enable such control. Input device 226 generates electronic instructions 
over input link 228 in response to preprogrammed data or in response to a viewer pressing 

^ buttons on input device 226. Input device 226 may also control Web browser software 

within management system 212 as when management system 212 is a set-top box or an 

m Internet terminal that has been adapted to perform the operations disclosed herein. For 

instance, input device 226 may be programmed to turn on home entertainment system 210 

P and to tune management system 212 to a channel. 

^ 0040 Figure 2 illustrates a signal recorder 230, which is capable of receiving video 

^ and/or audio data and recording the data on a storage medium. Video signals are 

stags 

transmitted to display device 214 and/or signal recorder 230 by video image links 232a and 
232b, respectively, examples of which include a radio-frequency ("RF") link, an S-video 

3 

$ ^ link, a composite link, or any other equivalent form of video image link. Similarly, audio 

^ H ^ | 3 00 

£q o5 links 234a and 234b transmit audio data from management system 212 to audio system 

S^lSo^ 216 and/or to signal recorder 230. 

Q ggg coo 



5s " 5 *5i 



Si^llS The operation of management system 212 is controlled by a central processing 

p 



unit ("CPU")> illustrated as processing unit 236, which is coupled to an application- 
^ specific integrated circuit ("ASIC") 238 via system bus 240 and uses computer-executable 

instructions implemented in software and/or hardwired logic circuitry. Processing unit 236 



and ASIC 238 are also coupled via a system bus 240 to various other system components, 
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including system memory 242, mass storage interface 244, user interface 224 and signal 
input 218. Processing unit 236 may execute software designed to implement features of 
management system 212 including features of the present invention. 

0042 ASIC 238 contains circuitry that is used to implement certain functions of 
management system 212. Instructions, data, and other program modules necessary for the 
operation of processing unit 236 and necessary for the operation of the ASIC 238 may be 
stored in mass storage device 250 and/or system memory 242, which includes read-only 

Q 

*fi memory ("ROM") 246 and random-access memory ("RAM") 248. System memory 242 is 

^ coupled to system bus 240 and mass storage device 250 is coupled to mass storage 

;*? interface 244, which is in turn also coupled to system bus 240. Thus, ROM 246, RAM 248 

r J* and mass storage device 250 are communicatively coupled to ASIC 238 so as to be 

3 readable by ASIC 238 and so that data may be written from ASIC 238 to RAM 248 and to 

<%j mass storage device 250. Mass storage device 250 may be a magnetic hard disk 252, but 

H may also be any of the other computer-readable media referenced above. 

0043 Any desired computer-readable instructions or data, including application 
programs 254, other program modules 256, and an electronic programming guide ("EPG") 

S 

g „ 258, which specifies the broadcast times and channels of programs can be stored in mass 

§ o 3 o I ffi storage device 250. 

S h ui h b 

O ° 00 < P ^ 

© i £ % B G 0044 Mass storage device 250 may also be used to record video data 253, in which 

Q § g j « o 

v-7 w 2 ^ j 5 

* 8 5 g S £ case, management system 212 performs the functions of a digital video recorder. Digital 

<; < % 

^ video data may be received by home entertainment system 2 1 0 from a variety of sources 

p including signal source 220, remote computer 260, video game 268, input device 226 and 

the Internet. 



- Page 14 - DocketNa 14531.101 



0045 EPG data may be obtained in a variety of manners. For instance, the EPG data 
can be supplied to management system 212 by a remote computer 260, such as a server, or 
from devices on the Internet and stored on mass storage device 250. The EPG data may be 
supplied on a regular basis to continually maintain a current schedule of programming at 
the management system 212. Alternatively, the EPG may be delivered to home 
entertainment system 210 by using a direct-dial communication over standard telephone 
lines, or by using data transmission over the cable television infrastructure, a satellite 
network, an over-the-air broadcast or any other available medium, including those 
■yj previously mentioned. 

CO 0046 In the embodiment where management system 212 is associated with the 

HF Internet, management system 212 may communicate with remote computer 260 via wide 

y area network ("WAN") 262 using a variety of techniques, including interposing serial port 

;;J'J interface 264 between the system bus 240 and a modem 266, using a wireless link, or other 

i2 means for establishing communications over a WAN that may be internal or external to 

management system 212. Management device 212 is also capable of transmitting 
information via the Internet by direct-dial communication over standard telephone lines, or 

w 

$ ^ by using any other available communication medium. 

§5 o S g 1 5 0047 While serial port interface 264 may be utilized to connect a modem 266 for 

05 ° v) < p o 

§ 1 1 2 5 § communicating across a WAN, serial port interface may also be utilized to connect other 

^ llSli 

-J«§S£ consumer electronic devices, such as video game 268, and/or various input devices, such 

<S* < < 

§ as a keyboard (not shown) or joystick (not shown), to management device 2 1 2, 



^ 0048 Referring now to signal input 218, if the signal on programming input line 222 

includes multiple channels, a tuner 270 included in signal input 218 tunes to a selected 
channel in the signal. Multiple tuners 270 can be used to provide enhanced viewing 
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features, such as picture-in-picture, recording one channel while viewing another, and 
recording a plurality of channels simultaneously, A signal decoder 272 may convert video 
data from an analog format to a digital format, from a digital format to an analog format, or 
convent between varying digital formats, in the event that ASIC 238 and tuner 270 employ 
different formats. Video decoder 272 may also decode video data from a compressed 
video format (e.g. MPEG). In embodiments where the management system 212 includes 
multiple tuners 270, management system 212 may also include multiple signal decoders 
272 to perform the operations disclosed herein, 
ifi 0049 Management system 212 may also include video output 274, which may 

;fi include a video converter that switches between analog and digital formats as necessary 

when providing video data over video links 232a and 232b. Similarly, audio output 276 
'5 can include an audio converter to provide the necessary switching between analog and 

^ digital formats across audio links 234a and 234b. 

r: 0050 While Figure 2 and the corresponding discussion above provide a general 

description of a suitable environment in which the invention may be implemented, it will 
^ be appreciated that the features of the present invention disclosed herein may be practiced 

_ in association with a variety of different system configurations. 

< & §: Sj 

w85S*S 0051 Figure 3 illustrates a network 300 in which a multimedia source 301 

O ° < 5 o 

§ t j| 2 o § communicates a plurality of multimedia packets 304 over heterogeneous networks 303 to a 

^ illli 

- S < 8 £ r! multimedia sink 302. The wide area network 262 described above with reference to Figure 

< < i 

§ 2 is one example of a heterogeneous network, with the multimedia source 301 being the 



^ remote computer 260, and the multimedia sink 302 being the home entertainment system 

210. However, the multimedia sink 302 may also be a personal digital assistant, a lap top 
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computer, a desk top computer, a telephone, or any other device capable of receiving 
multimedia packets and rendering in real-time the associated multimedia information. 
0052 The heterogeneous networks 303 may include three classifications of networks 
as follows: 

1 ) constant delay networks 
(e.g., constant delay network 311); 

2) variable delay networks that have a common time base 
(e.g., variable delay network 3 12 that has common time base 322); and 

r\ 3) variable delay networks that do not have a common time base 

m (e.g., variable delay network 313 that does not have a common time base). 

s jg 0053 Although the heterogeneous networks 303 are illustrated as having only one of 

Q each classification of networks, the principles of the present invention may be applied to 

M; heterogeneous networks that have zero or one or more of each classification of networks. 

The heterogeneous networks are illustrated such that multimedia packets 304 would 
traverse, in order, the constant delay network, the variable delay network with a common 
£j time base, and the variable delay network without a common time base. This 

w ^ configuration is provided for illustrative purposes only. The principles of the present 

pq 1 3 1 § § invention apply regardless of the specific order of each network within the routing path of 

O ° « < S O 

S 2 § £ a multimedia packet. 

~$^slrf 0054 In order to transition the multimedia packets from one network to the next, a 

< < 5 

§ gateway is provided between each network in the heterogeneous networks. For example, 



^ gateway 331 receives multimedia packets from constant delay network 311, performs 

appropriate functions on those packets as described herein, and provides those packets over 
the variable delay network 312. In addition, gateway 332 receives multimedia packets 
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from variable delay network 312, performs appropriate functions on those packets as 
described herein, and provides those packets over the variable delay network 313. The 
gateways receive multimedia packets from a previous network, reconfigure the multimedia 
packet if necessary to conform to the standards of the next network, and then transmit the 
potentially reconfigured multimedia packet onto the next network. 

0055 Figure 4 illustrates a method 400 of performing constant delay communication 
of a stream of multimedia packets over heterogeneous networks in accordance with the 
Q present invention. Figure 3 illustrates an initial state of the heterogeneous networks before 

M the method of Figure 4 commences. Figures 5 through 6 show successive states of the 

^ heterogeneous networks that result from the performance of the method of Figure 4, 

% Accordingly, the method of Figure 4 will be described with frequent reference to the 

jU network states illustrated in Figures 3, 5 and 6. 

|1 0056 Initially, a common time reckoning is established in those variable delay 

Q networks in the heterogeneous networks that do not already have a common network time 

base (act 401). Figure 5 illustrates the network state of the heterogeneous networks after 
>j act 401 is complete. The modified form of the heterogeneous networks 303 of Figure 3 are 

£j ^ illustrated as heterogeneous networks 503 in Figure 5. The modified form of the variable 

§3 oS g§3j delay network 313 of Figure 3 is illustrated as variable delay network 513 in Figure 5. 

© ^(328= Note that variable delay network 513 has a common time base 523, whereas the variable 

delay network 313 does not. At this stage, each of the networks in the heterogeneous 

< < 3 

§ networks 503 should either be a constant delay network, or a variable delay network that 



O 



has a common network time base. 

0057 Returning to method 400 of Figure 4, the variable delay networks are emulated 
as one or more constant delay networks using the common time reckonings followed by 
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each variable delay network (act 402). This may be accomplished by the link layer, the 
application layer or another layer in the gateway including a time stamp in the multimedia 
packet, the time stamp representing a time that follows the common time reckoning of the 
next variable delay network. The time stamp may be related to the time that the packet is 
to be rendered and may be, for example, the time that the packet is transmitted. The 
receiving device may then use the time stamp to determine when the packet should be 
displayed. The time stamp may also be used to determine what time the multimedia packet 
O should be transmitted by subsequent gateways, or perhaps what time future time stamps 

j £j should indicate for subsequent networks. Referring to Figure 5, the common time base 322 

^ of the variable delay network 312 is used to emulate the variable delay network 312 as a 

"J constant delay network. In addition, the common time base 523 of the variable delay 

n network 513 may be used to emulate the variable delay network 513 as a constant delay 

network. 

*3 0058 Figure 6 illustrates the network state of the heterogeneous networks after act 

402 is complete. The modified form of the heterogeneous networks 503 of Figure 5 are 
illustrated as heterogeneous networks 603 in Figure 6. The emulated constant delay 

3 

$ ^ network 612 replaces the variable delay network 312 to emphasize its constant delay 

& i & w ^ 

pqo3g§3j emulation. In addition, the emulated constant delay network 613 replaces the variable 

S^w2o£ delay network 513 to emphasize its constant delay emulation. Now the network is 

■Js§££ sufficiently prepared to transmit multimedia packets from the multimedia source and the 



* 3 ^ O o u. 



multimedia sink (act 403) in real-time. 



^ 0059 As a multimedia packet transitions from one network to the next in the 

heterogeneous networks, the gateway may reformat or reconfigure the multimedia packet 
to conform with the requirements of the next network if necessary. In some cases, this 

- Page 19 - Docket No. 14531.101 



may involve including a time stamp that conforms to the format of the next network and 
that represents an accurate time according to the time reckoning of the next network. Such 
time translation need not occur, however, if both the previous network and the next 
network recognize the same time reckoning, or if the next network is a constant delay 
network. 

0060 When transitioning from a constant delay network to a constant delay network, 
the gateway should ensure that the packet is transmitted onto the next network at a 
relatively constant time period after the packet was received from the previous network. 
This transition may be accomplished without interpreting, manipulating, or creating time 
stamps for the packet. 

0061 When transitioning from a constant delay network to a variable delay network, 
the gateway will receive the packet at a given receipt time from the constant delay 
network. The gateway will then generate a time stamp that represents a time according to 
the common time reckoning of the variable delay network, the time being equal to or 
having a relatively constant offset from the time that the packet was received from the 
constant delay network as measured using the common time reckoning of the variable 
delay network. 

0062 When transitioning from a variable delay network to a constant delay network, 
the gateway will receive the packet and interpret the time stamp according to the time 
reckoning of the variable delay network to determine the time (according to the time 
reckoning of the gateway) when the packet should be transmitted onto the constant delay 
network. The gateway will then transmit the packet onto the constant delay network at the 
given transmit time. 
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0063 When transitioning from a variable delay network to another variable delay 
network, the gateway will leave the time stamp in the multimedia packet if both variable 
delay networks follow the same time reckoning and packet format. Otherwise, the 
gateway will translate the time stamp so that it represents a time according to the common 
time reckoning of the second variable delay network. 

0064 In some variable delay networks, there is no provision for a link layer controller 
to maintain a common reckoning of time with other devices on the network. Accordingly, 

y the application layer itself may maintain the common time reckoning across the devices, 

calculate the appropriate time stamp, and/or include the time stamp in the multimedia 

^ packet. Which of these tasks the application layer performs may depend on the capabilities 

s g of the link layer that the application layer uses to forward multimedia packets through the 

O gateway. 

H* 0065 In this manner, even heterogeneous networks may emulate a constant delay 

network that allows for real-time communication of multimedia packets. Thus, the 
principles of the present invention provide for the delivery of multimedia packets 
regardless of the heterogenic nature of the networks that intervene between the multimedia 

si 

$ _ source and the multimedia sink, even if some of the heterogeneous networks include 



IX} 



§3 o 5 g S 5 variable delay network that do not themselves support a common notion of time. Thus, 



g 1 1 2 8 g streaming multimedia data may be available regardless of the location of the user. 

3* S.H 



Accordingly, the principles of the present invention represent a significant improvement 
over the state of the art. 



^ 0066 Various examples are now provided of how to emulate a constant delay 

network from a variable delay network. In particular, Figure 7 illustrates a network 
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configuration 700 that includes a transmitter 720 and a receiver 721 and a variable delay 
network 704 that intervenes between the transmitter 720 and the receiver 721 . 
0067 The transmitter 720 includes a transmitter application 701 that stores a plurality 
of multimedia packets in a memory 707. For example, memory 707 stores two multimedia 
packets, a first multimedia packet 708 and a second multimedia packet 709, The 
transmitter 720 also includes a transmitter link layer controller 703 that actually retrieves 
the multimedia packets for transmission over the variable delay network 704. A variable 
y delay interface 702 intervenes between the transmitter application 701 and the transmitter 

T\ link layer controller 703. The receiver 721 includes a link layer controller 705, which 

receives the multimedia packets from the variable delay network 704 and provides those 

I'M 

jg packets to the receiver application 706. 

CJ 0068 The variable delay network 704 may be, for example, an IEEE 1394 serial bus 

H network. It should be noted that the conventional IEC 61 883 -x technology allow for 

j*f constant network emulation of a IEEE 1394 serial bus network using an AV link layer 

controller that provides IEC 61883-x support. However, this conventional constant delay 

^ emulation relies on a constant delay between the transmitter application dispatching the 
w 

gj _ multimedia packets, and the transmitter link layer controller receiving the multimedia 

o 3 g 1 3j packet. However, as illustrated in Figure 7, there is a variable delay interface 703 between 

m P h 

S SwSoG the transmitter application 701 and the link layer controller 703. Therefore, there is no 

^ < I S t reason to believe that the conventional constant delay emulation methods using an AV link 

< < £ 

§ layer controller that provides IEC 61883-x support would work in the network 



^ configuration illustrated in Figure 7. 

0069 The variable delay interface 702 may be, for example, an undedicated PCI 
interface. In this case, the transmitter link layer controller 703 may be, for example, an 
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OHCI link layer controller. The PCI interface has the advantage of being able to be shared 
among a variety of devices, not just the transmitter link layer controller 703. Accordingly, 
the PCI interface is well suited for general purpose computing systems. However, 
conventional technology does not provide for constant delay network emulation of a 
variable delay network when the transmitter of the variable delay network includes a 
variable delay interface between the transmitter application and the transmitter link layer 
controller. The principles of the present invention, however, allow for a PCI interface with 
an OHCI link layer controller to be used in a transmitter while still emulating a constant 
delay network, 

0070 Figure 8 illustrates a method 800 of a transmitter link layer controller emulating 
a constant delay network over a variable delay network despite the undedicated variable 
delay interface in accordance with the present invention. The method 800 of Figure 8 will 
be described with respect to the network configuration 700 of Figure 7. 

0071 The transmitter link layer controller 703 receives a first multimedia packet 708 
from the transmitter application 701 (act 801). The transmitter application 701 included a 
first transmitter application time stamp 710 in the packet, the time stamp 710 representing 
the relative time that the information in the first multimedia packet 708 should be rendered 
by the receiver application 706 as represented in the time base followed by the transmitter 
application 701. The transmitter link layer controller 703 then includes another time stamp 
714 in the first multimedia packet 708 (act 802). The new time stamp represents the time 
that the corresponding information should be rendering with reference to the network time 
base. 

0072 The transmitter link layer controller 703 also receives information from the 
transmitter application 701 from which the transmitter link layer controller 703 may derive 
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the frequency (act 803) of the time base used by the transmitter application with respect to 
the common time base (i.e., the network time base) recognized over devices on the variable 
delay network 704. The transmitter link layer controller 703 stores this information as 
frequency 712, and also stores the current time 713 in accordance with the common time 
base recognized over the variable delay network 704. 

0073 The transmitter link layer controller also receives a second multimedia packet 
such as multimedia packet 709 from the transmitter application 701 over the variable delay 
interface 702 (act 804). The transmitter application 701 included a transmitter application 
time stamp 711 in the packet, the time stamp 711 representing the relative time that the 
information in the second multimedia packet 709 should be rendered by the receiver 
4* application 706, as represented in the time base followed by the transmitter application 

a 7ol 

^ 0074 The transmitter link layer controller 703 then calculates a network time stamp 

i 3 ?* 715 that represents the relative time that the information in the second multimedia packet 
should be rendered by the receiver application 706 in accordance with the common 

£j network time base (act 805). The calculation is based on the first transmitter application 

£3 ^ time stamp 710, the second transmitter application time stamp 71 1, the frequency 712, and 

pqi^gS^ the first network time stamp 714. More specifically, this calculation may be made 

g 3 £ 2 o £ according to the following formula: 

-ISsSh new_network_time = oldjietworkjime + (newJSTBjime - old_STB__time) * 



^ o © w 

* 2^ 



frequency 



^ where; 



old_networkJime = the stored value of the time stamped in the last packet; 
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old_STB_time = the stored value of the software time base provided in the last 
packet; 

new_STB _time = the value of the software time base read from the current packet; 
and 

frequency = the stored software time base frequency. 
In actual implementation, a fixed offset may be added to the network time stamps to be in 
compliance with any applicable specification such as IEC 61 883 -x. 
O 0075 The transmitter link layer controller 703 then includes the calculated network 

OS time stamp 715 in the second multimedia packet 709 (act 806), and then dispatches the 

JiJ second multimedia packet 709 over the variable delay network 704 (act 807). The 

% calculated network time stamp 715 may be included in the multimedia packet in 

p accordance with the IEC 61883-x protocol. The receiver link layer controller 705 may 

ik then receive the multimedia packets and inteipret the time stamp using the IEC 61883-x 

O protocol. The link layer controller 705 may then forward the packets to the receiver 

application 706 which may then time the rendering of the corresponding information. 
^ 0076 Figure 9 illustrates a network configuration 900 in which a transmitter 

3 

w 

$ „ application 901 controls the time base across a variable delay network 903. The 

§3 is transmitter application 901 is associated with a transmitter link layer controller 902 that 

O y w <1 h k" 

g % 5 2 g E stores a time base 913, A receiver application 905 is associated with a receiver link layer 

„oS§ou controller 904 that stores a time base 914, that is relatively synchronized with the time base 



< 



^ ^ o o h 



613. 



S£ 0077 Figure 10 illustrates a method 1000 for emulating a constant delay network 

over the variable delay network and will be described with respect to Figure 9. The 
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transmitter link layer controller 902 stores a transmitter application time base 913 received 
from the transmitter application 901 (act 1001). 

0078 The transmitter link layer controller periodically transmits the transmitter 
application time base to one or more devices including the receiver link layer controller 

904 over the variable delay network 903 (act 1002). Software that provides data 
asynchronously to the multimedia packets may perform the task of synchronization. In 
this manner, the clock registers at the transmitter and receiver link layer controllers (i.e., 
the time base 913 and the time base 914) may be kept synchronized. 

0079 The transmitter link layer controller 902 includes the transmitter application 
time base in each packet that is to be transmitted over the variable delay network (act 
1003) and then dispatches the packet over the network (act 1004). The receiver application 

905 may then use the transmitter application time base in order to time the rendering of the 
corresponding information in the multimedia packets. An example of a variable delay 
network in which the method 1000 of Figure 10 may be implemented is an IEC 802.11 
wireless network. 

0080 Therefore, the principles of the present invention provide for a means of 
emulating a constant delay network even over a group of heterogeneous networks that 
include variable delay networks that do not inherently support a common notion of time. 
Also, the principles of the present invention allow for constant delay network emulation on 
variable delay networks in which there is a variable delay interface between the transmitter 
application and the transmitter link layer controller, and in variable delay networks that 
have no inherent common network time base. 

0081 The present invention may be embodied in other specific forms without 
departing from its spirit or essential characteristics. The described embodiments are to be 
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considered in all respects only as illustrative and not restrictive. The scope of the 
invention is, therefore, indicated by the appended claims rather than by the foregoing 
description. All changes, which come within the meaning and range of equivalency of the 
claims, are to be embraced within their scope. 

0082 What is claimed and desired secured by United States Letters Patent is: 
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